Several statistical packages that do ANOVAs offer one or more post-hoc tests as optional

output, so programmers tend to request output for both ANOVAs and post-hoc tests, even before

they know whether the ANOVA is statistically significant or not, which can be confusing. ANOVA

output from other software can include a lot of extra information, such as a table of the mean,

variance, standard deviation, and count of the observations in each group. It may also include a

test for homogeneity of variances, which tests whether all groups have nearly the same SDs. In

R, the ANOVA output is very lean, and you have to request information like this in separate

commands.

Executing and interpreting post-hoc t tests

In the previous example, the ANOVA was statistically significant, so it qualifies for post-hoc pairwise

t tests. Now that we are at this step, we need to select which adjustment to use. We already have an

idea of what would happen if we used the Bonferroni adjustment. We’d have to run t tests like we did

before, only this time we’d have to use the three-level MARITAL variable and run three t tests: One

with M and NM, a second with M and OTH, and a third with M and OTH. For each p value we got, we

would have to compare it to the adjusted Bonferroni α of 0.016 instead of 0.05. By evaluating each p

value, you can determine which pairs of groups are statistically significantly different using the

Bonferroni adjustment.

But Bonferroni is not commonly used in statistical software. In R, the most common post-hoc

adjustments employed are Tukey-Kramer (using the TukeyHSD command) and Scheffe (using the

ScheffeTest command from the package DescTools). The reason why the Tukey HSD is not available in

R is that the Tukey-Kramer can handle both balanced and unbalanced groups. In the case of marital

statuses and fasting glucose levels in NHANES, the Tukey-Kramer is probably the most appropriate

test because we do not need the special features of the Scheffe test. However, we explain the output

anyway so that you can understand how to interpret it.

To run the Tukey-Kramer test in R, we use the following code: TukeyHSD(GLUCOSE_aov,

conf.level=.95). Notice that the code refers to the ANOVA object we made previously called

GLUCOSE_aov. The Tukey-Kramer output begins by restating the test, and the contents of the ANOVA

object GLUCOSE_aov.

Next is a table (also known as a matrix) with five columns. The first column does not have a heading,

but indicates which levels of MARITAL are being compared in each row (for example, 2-1 means that

1 = M is being compared to 2 = NM). The column diff indicates the mean difference between the

groups being compared, with lwr and upr referring to the lower and upper 95 percent confidence

limits of this difference, respectively. (R is using the 95 percent confidence limits because we

specified conf.level = .95 in our code.) Finally, in the last column labeled p adj is the p value for each

test. As you can see by the output, using the Tukey-Kramer test and α = 0.05, M and NM are

statistically significantly different (p = 0.0000102), and OTH and M are statistically significantly

different (p = 0.0030753), but NM and OTH are not statistically significantly different (p =

0.1101964).